Multiple Source-Detector Pair Photoplethysmography (PPG) Sensor

ABSTRACT

Systems, devices, and methods for tracking one or more physiological metrics (e.g., heart rate, blood oxygen saturation, and the like) of a user are described. For example, one or more light sources and one or more light detectors may be positioned on a wearable device such that light can be emitted towards the user&#39;s skin and further such that light reflected back to the wearable device can be measured and used to generate values for the one or more physiological metrics.

RELATED APPLICATION

This application is continuation-in-part of U.S. application Ser. No.15/948,970, filed Apr. 9, 2018, which claims the benefit of U.S.Provisional Application No. 62/482,997, filed on Apr. 7, 2017. Applicantclaims priority to and the benefit of each of such applications andincorporate all such applications herein by reference in its entirety.

TECHNICAL FIELD

This disclosure relates to the field of wearable devices, andparticularly to techniques for using photoplethysmography (PPG) sensorsto generate heart rate (HR) and other physiological metrics.

BACKGROUND

A PPG sensor may be utilized to detect the volumetric change in bloodvessels. A PPG sensor usually includes a light source, typically alight-emitting diode (LED), and a light-sensitive sensor, typically aphotodiode. Blood passing through the vasculature between the lightsource and the sensor will modulate the light path between the two,resulting in a deviation in the current produced by the photodiode. Byapplying various algorithms to the signal sensed by the photodiode, anHR estimate can be determined.

Further, by looking at signals corresponding to two or more wavelengths(e.g., red and infrared), a pulsatile blood oxygenation estimate (SpO2)can be obtained. SpO2 refers to a fraction of oxygen-saturatedhemoglobin relative to total hemoglobin in the blood. Decreased SpO2 inthe blood can lead to impaired mental function, or loss ofconsciousness, and may serve to indicate other serious healthconditions, such as sleep apnea or cardiovascular disease. Therefore,accurately measuring SpO2 is important in certain kinds of healthmonitoring.

Typical PPG technologies rely on emitting wavelengths of green, red,and/or infrared (IR) light from an LED. Many wearable PPG devices usegreen light, as the hemoglobin absorption of light is up to 20 timesgreater at green wavelengths than at IR wavelengths. Additionally, insome cases, green LED light sources may provide superior results interms of cost, form factor, and power efficiency.

SUMMARY

The systems, methods, and devices of this disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

In an aspect, the present disclosure is directed to a method forgenerating a physiological metric. The method includes obtaining a firstPPG signal based on light received by a light detector from a firstlight source during a first temporal window during which a second lightsource is not emitting light. The method also includes obtaining asecond PPG signal based on light received by the light detector from thesecond light source different from the first light source and during asecond temporal window that is different from the first temporal windowand during which the first light source is not emitting light. Further,the method includes removing a motion component from at least one of thefirst and second PPG signals based on information associated with afirst distance between the first light source and the light detector anda second distance between the second light source and the lightdetector. Moreover, the method includes generating the physiologicalmetric based on at least one of the first PPG signal and the second PPGsignal having the motion component removed therefrom. In addition, themethod includes causing the physiological metric to be displayed via auser interface on a user device.

In another aspect, the present disclosure is directed to a method forgenerating a heart rate estimation for a user device. The methodincludes obtaining multiple photoplethysmography (PPG) signals based onlight received by at least one light detector of the user device frommultiple light sources during different temporal windows during whichother light sources of the multiple light sources are not emittinglight, wherein the multiple PPG signals each represent a differentchannel of light received by the at least one light detector from themultiple light sources. The method also includes processing the multiplePPG signals to produce an output corresponding to a heart rateprobability distribution for the multiple PPG signals. Further, themethod includes determining the heart rate estimation inbeats-per-minute using the output corresponding to the heart rateprobability distribution for the multiple PPG signals. Moreover, themethod includes causing the heart rate estimation to be displayed via auser interface on the user device.

In yet another aspect, the present disclosure is directed to a wearabledevice for tracking a heart rate estimation of a user. The wearabledevice includes a plurality of light sources, a light detector, amemory, and one or more processors. The memory storescomputer-executable instructions for causing the one or more processorsto obtain multiple photoplethysmography (PPG) signals based on lightreceived by the light detector from the multiple light sources duringdifferent temporal windows during which other light sources of themultiple light sources are not emitting light, process, via a trainedmachine learning model of the one or more processors, the multiple PPGsignals to produce an output corresponding to a heart rate probabilitydistribution for the multiple PPG signals, determine the heart rateestimation in beats-per-minute using the multiple PPG signals and theoutput corresponding to the heart rate probability distribution for themultiple PPG signals, and cause the heart rate estimation to bedisplayed via a user interface on the wearable device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example monitoring device that may be worn by auser in accordance with aspects of this disclosure.

FIG. 1B illustrates an example hardware architecture of a monitoringdevice in accordance with aspects of this disclosure.

FIGS. 1C-1I illustrate example arrangements of light sources and lightdetectors in accordance with aspects of this disclosure.

FIG. 2 illustrates a perspective view of a monitoring device inaccordance with aspects of this disclosure.

FIG. 3 illustrates an example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure.

FIG. 4 illustrates another example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure.

FIG. 5 is an example block diagram of a system used for determining HRestimate in accordance with aspects of this disclosure.

FIG. 6 illustrates another example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure.

FIG. 7 illustrates another example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure.

FIG. 8 illustrates an example method of generating a heart rateestimation on a user device in accordance with aspects of thisdisclosure.

FIG. 9 illustrates another example arrangement of light sources andlight detectors for the PPG device in accordance with aspects of thisdisclosure.

FIG. 10 illustrates a schematic diagram of a computing system having amachine learning model for generating a heart rate estimation on a userdevice in accordance with aspects of this disclosure.

FIG. 11 illustrates a schematic diagram of a neural network for amachine learning model for generating a heart rate estimation on a userdevice in accordance with aspects of this disclosure.

FIG. 12 illustrates a schematic diagram of a neural network architecturefor a machine learning model for generating a heart rate estimation on auser device in accordance with aspects of this disclosure.

FIG. 13A illustrates a graph of a combination of heart rate estimatesand their corresponding probabilities from multiple channels/light pathsat one time interval in accordance with aspects of this disclosure.

FIG. 13B illustrates a graph of a single estimated heart rate with anoverall higher probability than the combination of the heart rateestimates and their corresponding probabilities from FIG. 13A inaccordance with aspects of this disclosure.

FIG. 14 illustrates a graph of heart rate estimate (y-axis) versus time(x-axis), particularly illustrating the heart rate probabilitydistributions as output by the machine learning model as well as thecombined heart rate estimate in accordance with aspects of thisdisclosure.

DETAILED DESCRIPTION

Existing PPG sensors can provide an effective method for measuring auser's HR. As noted above, a PPG sensor usually includes a light source,typically an LED, and a light-sensitive sensor, typically a photodiode.Using the photodiode, a PPG device including such a PPG sensor canobtain PPG signals indicative of how the blood is passing through thevessels and generate HR estimates based on the PPG signals.

Some PPG technologies rely on detecting light at a single spatiallocation, or adding signals taken from two or more spatial locations.Both of these approaches result in a single spatial measurement fromwhich the HR estimate (or other physiological metrics) can bedetermined. In some embodiments, a PPG device employs a single lightsource coupled to a single detector (i.e., a single light path).Alternatively, a PPG device may employ multiple light sources coupled toa single detector or multiple detectors (i.e., two or more light paths).In other embodiments, a PPG device employs multiple detectors coupled toa single light source or multiple light sources (i.e., two or more lightpaths). In some cases, the light source(s) may be configured to emit oneor more of green, red, and/or infrared light. For example, a PPG devicemay employ a single collimated light source and two or more lightdetectors each configured to detect a specific wavelength or wavelengthrange. In some cases, each detector is configured to detect a differentwavelength or wavelength range from one another. In other cases, two ormore detectors configured to detect the same wavelength or wavelengthrange. In yet another case, one or more detectors configured to detect aspecific wavelength or wavelength range different from one or more otherdetectors). In embodiments employing multiple light paths, the PPGdevice may determine an average of the signals resulting from themultiple light paths before determining an HR estimate or otherphysiological metrics. Such a PPG device may not be able to resolveindividual light paths or separately utilize the individual signalsresulting from the multiple light paths.

In some cases, if the user wearing the PPG device is performing anactivity involving motion (or contorting of the wrist, for example, fora wrist-worn PPG device, thereby affecting the dynamics of the bloodflow within the wrist), the accuracy of the HR estimate provided by thePPG device may be reduced or compromised. The light intensity receivedby the light detectors may be modulated by these movements typically atan order of magnitude or greater than the desired cardiac signal.Therefore, a preprocessing step where the signal effect of thesemovements is removed would be desirable and improve HR estimationaccuracy during motion.

In addition to the deleterious effects of motion, another cause ofreduced signal quality in PPG devices may be the characteristics of thelocal area being sensed. For instance, signal quality can varydramatically if a wrist-worn PPG sensor is moved only a few millimetersup or down the wrist. In addition, during motion, certain portions ofthe wrist-worn PPG devices may be subject to more motion depending ontheir location, position, and/or orientation, and PPG sensors placed onsuch portions may therefore result in greater degradation of the PPGsignal due to motion.

Overview of Improved Techniques for Measuring HR and Other PhysiologicalMetrics

Various embodiments of the present disclosure allow a PPG device toutilize signals based on two or more independently addressablesource-detector combinations such that the signal quality of the PPGdevice is improved, especially during activities involving motion. Insome embodiments, PPG signals can be acquired via multiple light pathsinvolving one or more sources and one or more detectors placed atdifferent spatial locations. These multiple PPG signals are thenprocessed to isolate the cardiac component (e.g., by removing the motioncomponent) from the PPG signals. For example, the motion component maybe removed based on inputs from the accelerometer, unsupervised learningand/or previously done supervised learning. Additionally, oralternatively, the PPG signals corresponding to these multiple lightpaths are compared using a quality metric such that the highest-qualityPPG signal can be selected to be used for estimating HR or otherphysiological metrics.

To utilize two or more source-detector pairs for motion signalrejection, a PPG device can use a computer program to identify themotion component of a given signal and remove the motion component fromthe composite signal, leaving only the cardiac signal as a remainder. Insome implementations, the temporal phase of the cardiac waveform isassumed to stay constant between different light paths, while the phaseof the motion signal is expected to vary between light paths due to howthe PPG sensor interacts with the skin surface during activitiesinvolving motion (e.g., pressure at the PPG/skin interface may varydepending on the spatial location of the light source and the lightdetector of the light path). Using this concept, PPG devices can fitmathematical models to the spatial light path signals to identify thecardiac and motion components. First, PPG signals are extracted by eachsource-detector combination. For example, two light sources and twolight detectors would result in four source-detector combinations. Amathematical model is then fit to the different spatial points, fromwhich characteristic signals are extracted related to the cardiac andmotion components of the PPG signals. Such techniques are described ingreater detail below with reference to FIG. 6. PPG devices may alsoimplement other techniques including, but not limited to, independentcomponent analysis (ICA) and other forms of blind source separation.

Although some embodiments are described with reference to HR or cardiaccomponents of PPG signals, the techniques described herein may beextended to other types of physiological metrics described herein (e.g.,SpO2) or other types of signals that can be extracted from the PPGsignals to determine such physiological metrics. For example, in someembodiments, a method for determining an SpO2 value comprises receivinga first set of one or more PPG signals from one or more PPG sensors,which may include analog signals or digital data sampled from analogcomponents and stored in computer memory. The first set of PPG signalsmay correspond to red and/or infrared light previously emitted by one ormore light sources 102 after the emitted light has interacted with theuser's skin, when the monitoring device is worn by the user. The firstset of PPG signals may include a noise component. The method fordetermining the SpO2 value may further comprise receiving a second setof one or more PPG signals from the one or more PPG sensors, which mayinclude analog signals or digital data sampled from analog componentsand stored in computer memory. For example, the second set of PPGsignals may be obtained from different ranges of wavelengths emittedfrom the light source 102 than the first set of PPG signals. Forexample, the second set of PPG signals may be obtained from one or moregreen light sources 102. In some cases, the second set of PPG signals isobtained from a system within the device used for tracking a user's HR.In other cases, the second set of PPG signals is received from a systemseparate from HR detection. The method for determining the SpO2 valuemay further comprise filtering the first set of PPG signals based on afeature of the second set of PPG signals to generate a filtered set ofPPG signals. Various filtering techniques may be used, depending onembodiments, to remove noise or other features from the first set of PPGsignals based on a feature of the second set of PPG signals. As oneexample, HR may be the feature of the second set of PPG signals. In thecase of HR, the device may create a filter based the detected frequencyof the HR signal. Examples of filters include a low-pass filter, ahigh-pass filter, and a narrow band filter that excludes frequenciesthat are inconsistent with the frequency of the HR signal. The methodfor determining the SpO2 value may further comprise using one range ofwavelengths to better measure an underlying signal on which thewavelengths of the first set of PPG signals operates. Based on thisunderlying signal (or features derived therefrom), the device canimprove the first set of PPG signals based on filtering noise from thefirst set of PPG signals. Further, the filtered set of PPG signals canbe used to create and store a SpO2 value. As an example, the filteredset of PPG signals may have a reduced or eliminated noise component andtherefore may serve as a more accurate basis for creating and storingthe SpO2 value.

According to some implementations, an intermediate HR estimation isperformed based on PPG signals from two or more light paths. For each ofthe acquired PPG signals, the PPG device may determine an estimate ofthe HR in beats-per-minute (BPM) and compute a confidence metricassociated with the PPG signal, which is indicative of the signalquality for the particular light path associated with the PPG signal. Itmay also be possible to compute a confidence metric without anintermediate HR estimation, for example by characterizingcharacteristics (e.g., statistics) of the PPG signal or filteredversions of the PPG signal. In some embodiments, each confidence metriccorresponds to a single PPG signal. In other cases, each confidencemetric corresponds to multiple PPG signals. For example, a confidencemetric may be computed for each way of combining the PPG signals (e.g.,signals A+B, signals A+C, signals B+C, signals A+B+C, etc.), as well asfor various combinations of PPG signals (e.g., selecting at least two ofsignals A, B, and C). In other cases, one confidence metric correspondsto a single PPG signal and another confidence metric corresponds to acombination of multiple PPG signals. The PPG device can select an HRestimate from the multiple HR estimates corresponding to the multiplelight paths (e.g., by selecting the HR estimate of the PPG signal havingthe highest confidence metric). Alternatively, the PPG device may assigndifferent weight values to the multiple HR estimates based on theconfidence metric values associated with the individual and/or multiplePPG signals and compute a final HR estimate based on the weight values.The confidence values and/or the weight values may be updated oroptimized using unsupervised machine learning. The PPG device mayimplement hysteresis logic which prevents jumping between light paths ina short time window if the confidence metric values corresponding to thetwo light paths are within a threshold value. The PPG device may alsoimplement logic configured to bias the selection of HR estimates basedon user data, activity data, movement data, or other data accessible bythe PPG device. The PPG device may apply a smoothing filter on the HRestimates, for example, to improve accuracy and provide a better userexperience. Such techniques are described in greater detail below withreference to FIG. 7.

Advantage

One advantage in some of the embodiments described herein is that thespatial information associated with the light sources and/or lightdetectors can be used by different algorithms to improve HR or otherphysiological metric estimation accuracy of the PPG sensing device,especially when the user of the device is exercising or performingactivities involving motion. Existing implementations typically rely onalgorithms to improve the HR or other physiological metric estimationperformance, but do not have the benefit of the extra sensor datagenerated based on multiple light paths.

Example PPG Device

Embodiments of the present disclosure provide PPG-based devices thatutilize multiple source-detector pairs to provide more accurate HR andother physiological metrics. As shown in FIG. 1A, a PPG device 100 (alsoreferred to herein as a monitoring device or a wearable device) worn bya user may include a plurality of light sensors 102 and a plurality oflight detectors 106. As indicated by the dashed arrows, light emittedfrom the light sensors 102 can be reflected back to the light detectors106. Although FIG. 1A shows an example in which the user is wearing thePPG device 100 on the inner wrist, in other embodiments, the PPG device100 may be worn on the outer wrist or side wrist or in locations such asthe ear, fingertips, ankle, neck, upper arm, torso, leg and/or forehead(e.g., such that light sources of the PPG devices are adjacent to bloodvessels of a human).

Light Path

For purposes of this disclosure, the term “light path,” in addition tohaving its ordinary meaning, refers to the probabilistic path of photonsfrom one location to another, typically from the light source (oremitter) to the light detector (or sensor). Photons emitted by the lightemitter will follow many different paths to each detector. Forsimplicity and clarity, the path that results from the opticalpower-weighted average of all the possible paths is described simply asthe “light path” in some embodiments. In some alternative embodiments,“light path” refers to the path along which most of the photons travel.In yet other embodiments, “light path” refers to an approximated vectorhaving an origin at a center of a light source and terminating anywherein the surface area of a detector, and representing an approximate pathof light from the source to the detector.

As described above, a light path represents an approximate path of lightfrom a given source to a given detector. Thus, for example, if there aremultiple sources 102 and multiple detectors 106, then a distinct lightpath exist between each of the multiple sources and each of the multipledetectors. Thus, consistent with the embodiments described herein, PPGsignals associated with any of the aforementioned light paths may beselectively obtained and utilized for estimating HR and/or otherphysiological metrics. For example, the PPG signals corresponding to anyof multiple paths may be compared using a quality/confidence metric suchas a signal-to-noise ratio (SNR), and the PPG signal having the highestquality can be selected to be used for estimating the HR and/or otherphysiological metrics.

For example, FIG. 1A illustrates two light sources 102 and five lightdetectors 106. Each light source 102 has a light path leading to each ofthe light detectors 106. Thus, the example of FIG. 1A has ten uniquelight paths. In other embodiments, the PPG device 100 may contain anynumber of light sources 102 and light detectors 106. In an embodiment,light sources 102 and light detectors 106 are configured to be alignedproximate to a user's skin when PPG device 100 is worn. “Proximate” maymean any of slightly separated from, near, adjacent to, or in directcontact with, but direct contact is not required. For example, in FIG.1A, the PPG device 100 is worn on the wrist of the user such that lightsources 102 and light detectors 106 are adjacent to the dorsal side ofthe wrist of the user (e.g., the side of the wrist facing the samedirection as the back of the hand). The positioning of the PPG device100 as shown in FIG. 1A is provided merely as an example, and otherembodiments may use alternative positioning. For example, the PPG device100 may be positioned such that light sources 102 and light detectors106 are proximate to the volar side of the wrist of the user (e.g., theside of the wrist facing the same direction as the palm of the hand)when the PPG device 100 is worn by the user.

PPG Device Architecture

With reference to FIG. 1B, an example hardware architecture of the PPGdevice 100 is described. As shown in FIG. 1B, the PPG device 100comprises one or more light sources 102 and one or more light detectors106. The PPG device 100 may further comprise one or more processors 110(also referred to herein as processor 110) coupled to memory 112,display 114, bus 116, one or more input/output (I/O) devices 120, andwireless networking interface 124. Display 114 and/or I/O devices 120may be omitted in certain embodiments. Display 114 may comprise a liquidcrystal display, light-emitting diode display, touchscreen, or otherelectronic digital display device. Display 114 may be programmed orconfigured to display data, such as HR, blood oxygen saturation (SpO2)levels, and other metrics of the user. For example, the processor 110may compute values for the physiological metrics monitored by the PPGdevice 100 based on one or more PPG signals generated by the lightdetectors 106. In an embodiment, the PPG device 100 is a wristband andthe display 114 is configured such that the display faces away from theoutside of a user's wrist when the user wears the PPG device. In otherembodiments, the display 114 may be omitted and data detected by the PPGdevice 100 may be transmitted using the wireless networking interface124 via near-field communication (NFC), Bluetooth, Wi-Fi, or othersuitable wireless communication protocols to a host computer 130 foranalysis, display, and/or reporting.

I/O devices 120 may include, for example, motion sensors, vibrationdevices, lights, loudspeakers or sound devices, microphones, or otheranalog or digital input or output devices. For example, in addition tothe elements shown in FIG. 1B, the PPG device 100 may include one ormore of biometric sensors, optical sensors, inertial sensors (e.g.,accelerometer, gyroscope, etc.), barometric sensors (e.g., altimeter,etc.), geolocation sensors (e.g., GPS receiver), and/or other sensor(s).

Memory 112 may comprise RAM, ROM, FLASH memory, or other digital datastorage, and may include a control program 118 comprising sequences ofinstructions which, when loaded from the memory and executed using theprocessor 110, cause the processor 110 to perform the functions that aredescribed herein. Light sources 102 and light detectors 106 may becoupled to bus 116 directly or indirectly using driver circuitry 122 bywhich the processor 110 may drive the light sources 102 and obtainsignals from the light detectors 106.

The host computer 130 may be coupled to the wireless networkinginterface 124 via one or more networks 128, which may include one ormore local area networks, wide area networks, and/or internetworks usingany of terrestrial or satellite links. In some embodiments, hostcomputer 130 executes control programs and/or application programs thatare configured to perform some of the functions described hereinincluding but not limited to the processes described herein with respectto FIGS. 3-7.

In some embodiments, each light source 102 (e.g., light sources A, B, C,D) can be individually controlled, or each light detector 106 (e.g.,light detectors E, F, G, H) can be individually read out when multipledetectors are used, and in such embodiments, PPG sensor data alongseveral different light paths can be collected. For example, in thearrangement shown in FIG. 1B, if the control program 118 drives thelight sources A-D one at a time, by the time the light sources A-D haveeach been activated once, each of the light detectors E-H would havecaptured four sets of data (e.g., one for each light source), resultingin 16 sets of data based on the 16 light paths. The control program 118can utilize the collected data to provide a more accurate estimation orHR and/or other physiological metrics.

In related aspects, the processor 110 and other component(s) of the PPGdevice 100 may be implemented as a System-on-Chip (SoC) that may includeone or more central processing unit (CPU) cores that use one or morereduced instruction set computing (RISC) instruction sets, and/or othersoftware and hardware to support the PPG device 100.

Other Functions of PPG Device

The PPG device 100 may collect one or more types of physiological and/orenvironmental data from one or more sensor(s) and/or external devicesand communicate or relay such information to other devices (e.g., hostcomputer 130 or another server), thus permitting the collected data tobe viewed, for example, using a web browser or network-basedapplication. For example, while being worn by the user, the PPG device100 may perform biometric monitoring via calculating and storing theuser's step count using one or more sensor(s). The PPG device 100 maytransmit data representative of the user's step count to an account on aweb service (e.g., www.fitbit.com), computer, mobile phone, and/orhealth station where the data may be stored, processed, and/orvisualized by the user. The PPG device 100 may measure or calculateother physiological metric(s) in addition to, or in place of, the user'sstep count. Such physiological metric(s) may include, but are notlimited to: energy expenditure, e.g., calorie burn; floors climbedand/or descended; HR; heartbeat waveform; HR variability; HR recovery;respiration, SpO2, blood volume, blood glucose, skin moisture and skinpigmentation level, location and/or heading (e.g., via a GPS, globalnavigation satellite system (GLONASS), or a similar system); elevation;ambulatory speed and/or distance traveled; swimming lap count; swimmingstroke type and count detected; bicycle distance and/or speed; bloodglucose; skin conduction; skin and/or body temperature; muscle statemeasured via electromyography; brain activity as measured byelectroencephalography; weight; body fat; caloric intake; nutritionalintake from food; medication intake; sleep periods (e.g., clock time,sleep phases, sleep quality and/or duration); pH levels; hydrationlevels; respiration rate; and/or other physiological metrics.

The PPG device 100 may also measure or calculate metrics related to theenvironment around the user (e.g., with one or more environmentalsensor(s)), such as, for example, barometric pressure, weatherconditions (e.g., temperature, humidity, pollen count, air quality,rain/snow conditions, wind speed), light exposure (e.g., ambient light,ultra-violet (UV) light exposure, time and/or duration spent indarkness), noise exposure, radiation exposure, and/or magnetic field.Furthermore, the PPG device 100 (and/or the host computer 130 and/oranother server) may collect data from one or more sensors of the PPGdevice 100, and may calculate metrics derived from such data. Forexample, the PPG device 100 (and/or the host computer 130 and/or anotherserver) may calculate the user's stress or relaxation levels based on acombination of HR variability, skin conduction, noise pollution, and/orsleep quality. In another example, the PPG device 100 (and/or the hostcomputer 130 and/or another server) may determine the efficacy of amedical intervention, for example, medication, based on a combination ofdata relating to medication intake, sleep, and/or activity. In yetanother example, the PPG device 100 (and/or the host computer 130 and/oranother server) may determine the efficacy of an allergy medicationbased on a combination of data relating to pollen levels, medicationintake, sleep and/or activity. These examples are provided forillustration only and are not intended to be limiting or exhaustive.

Source-Detector Arrangement

As discussed above, PPG devices according to the embodiments of thepresent disclosure includes multiple light sources and/or lightdetectors. FIGS. 1C-1I illustrate several example configurations oflight sources 102 and light detectors 106. Other configurations ofmultiple sources and/or multiple detectors may be implemented, forinstance by combining some of the configurations illustrated in FIGS.1C-1I. In some cases, a light source 102 described herein may includetwo or more co-located light emitters that are each configured to emitlight having a different center wavelength (e.g., a green light emitter,a red light emitter, and an infrared light emitter, or any combinationthereof, may be packaged in a single light source 102). Such co-locatedlight emitters may be packaged to be within less than 1 mm from eachother. Alternatively or additionally, a light source 102 describedherein may include a single light emitter configured to emit lighthaving a single center wavelength (e.g., a first light source 102 may beconfigured to emit just green light, a second light source 102 may beconfigured to emit just red light, and a third light source 102 may beconfigured to emit just infrared light). Further, a light detector 106described herein may be configured to detect light having a specificrange of wavelengths (e.g., green light, red light, or infrared light).Alternatively or additionally, a light detector 106 described herein maybe configured to detect light of multiple ranges of wavelengths (e.g.,green light, red light, and infrared light, or any combination thereof).

FIG. 1C illustrates a configuration in which five light detectors arearranged in a line between two light sources 102 (e.g., in aone-dimensional layout). Light emitted by the light sources 102 arereflected towards the light detectors 106 disposed between the two lightsources 102. FIG. 1D illustrates a configuration in which five lightsources 102 are arranged in a line between two light detectors 106(e.g., in a one-dimensional layout). Light emitted by the light sources102 located between the light detectors 106 are reflected towards thelight detectors 106. In some embodiments, the PPG device 100 includestwo or more light detectors between the light sources. Alternatively, oradditionally, the PPG device may include two or more light sourcesbetween the light detectors. The center points of all of the lightsources and light detectors may be aligned along a straight line (e.g.,in a one-dimensional layout).

FIG. 1E illustrates a configuration in which four light detectors 106are arranged in a box shape (e.g., in a two-dimensional layout) betweenthe two light sources 102. FIG. 1F illustrates a configuration in whichfour light sources 102 are arranged in a box shape (e.g., in atwo-dimensional layout) between the two light detectors 106.Alternatively, a single detector 106 may be surrounded by multiplesources 102 (e.g., in a ring geometry), or a single source 102 may besurrounded by multiple detectors 106 (e.g., in the ring geometry). Insome embodiments, the PPG device 100 includes a two-dimensional array oflight detectors located between two light sources. Alternatively, oradditionally, the PPG device 100 may include a two-dimensional array oflight sources located between two light detectors.

FIG. 1G illustrates a configuration in which two light sources 102 andtwo light detectors 106 are arranged in an alternating manner in a boxshape (e.g., in a two-dimensional layout). In some embodiments, the PPGdevice includes a two-dimensional array of light sources and lightdetectors, where the two-dimensional array includes the same number oflight sources as the light detectors. Additionally, or alternatively,the PPG device 100 includes a two-dimensional array of light sources andlight detectors, in which (i) the light sources are adjacent only to thelight detectors along the horizontal direction and the verticaldirection and (ii) the light detectors are adjacent only to the lightsources along the horizontal direction and the vertical direction, wherethe horizontal direction is parallel to the surface of the user's skinand the direction in which the user's arm extends when the device isworn by the user, and the vertical direction is parallel to the surfaceof the user's skin and perpendicular to the direction in which theuser's arm extends when the device is worn by the user. In some otherembodiments, the PPG device 100 includes a two-dimensional array oflight sources in light detectors, in which (i) the light sources areadjacent only to the light detectors along the horizontal direction andonly to other light sources along the vertical direction and (ii) thelight detectors are adjacent only to the light sources along thehorizontal direction and only to other light detectors along thevertical direction, where the horizontal direction is parallel to thesurface of the user's skin and the direction in which the user's armextends when the device is worn by the user, and the vertical directionis parallel to the surface of the user's skin and perpendicular to thedirection in which the user's arm extends when the device is worn by theuser.

FIG. 1H illustrates a configuration in which three light sources 102 arearranged along a vertical line or axis, and in which four lightdetectors 106 (e.g., two inner light detectors 106, closer to the centerlight source 102 and two outer light detectors 1060 that are fartheraway from the center light source 102) and the center light source 102are arranged along a horizontal line or axis. In one example, the centerlight source 102 includes a green light emitter, a red light emitter,and an infrared light emitter, the top and bottom light sources 102 eachinclude a green light emitter, the inner two light detectors 106, areconfigured to detect green light, and the outer two light detectors 1060are configured to detect red and infrared light. In other examples, theindividual light sources 102 and light detectors 106 include othercombinations of light sources or emitters.

In some embodiments, the light sources and detectors are arranged suchthat the distance between a green light source and a green lightdetector is shorter than the distance between a red or infrared lightsource and a red or infrared light detector. For example, as shown inFIG. 1H, the distance between the center light source 102 and the innerlight detectors 106, may be shorter than the distance between the centerlight source 102 and the outer light detectors 1060. In otherembodiments, the light sources and detectors are arranged such that thedistance between a green light source and a green light detector isgreater or equal to the distance between a red or infrared light sourceand a red or infrared light detector.

FIG. 1I illustrates a configuration in which five light sources 102 andfour light detectors 106 are arranged in the shape of a cross, each legof the cross having a light detector 106 and a light source 102, in thatorder, along a radial direction from the center of the cross, and alight source 102 being positioned at the center of the cross. In oneexample, the center light source 102 includes a green light emitter, ared light emitter, and an infrared light emitter, the left and rightlight sources 102 each include a red light emitter and an infrared lightemitter, the top and bottom light sources 102 each include a green lightemitter, the left and right light detectors 106 are configured to detectgreen light, and the top and bottom light detectors 106 are configuredto detect red and infrared light. In other examples, the individuallight sources 102 and light detectors 106 include other combinations oflight sources or emitters. In the example of FIG. 1I, light emitted bythe top, center, and/or bottom light sources 102 and received by the topand/or bottom light detectors 106 would be along one rotational axis(e.g., first rotational axis) of the device, and light emitted by theleft, center, and/or right light sources 102 and received by the leftand/or right light detectors 106 would be along another rotational axisof the device (e.g., second rotational axis, which, in this example, isorthogonal to the first rotational axis). In other implementations,additionally or alternatively, two or more axes that are not orthogonalto each other may be utilized (e.g., at an angle other than 90 degrees).

Although not shown in FIGS. 1C-1I, the PPG device 100 may include otherarrangements. For example, the PPG device 110 may include a single greenlight source and multiple light detectors (e.g., two, three, or morelight detectors). As another example, the PPG device 110 may include ared light source, a single infrared light source, and two red andinfrared light detectors. In some cases, one or more of the elementsshown in FIGS. 1C-1I may be omitted. Further, in some implementations,one or more of the arrangements of FIGS. 1C-1I may be flipped(horizontally or vertically) or rotated by a specific degree value(e.g., 30 degrees, 45, degrees, 60 degrees, 90, degrees, 180 degrees,etc, along either a clockwise direction or a counterclockwisedirection).

Perspective View of Example Wearable Device

FIG. 2 illustrates a schematic perspective view of a PPG device in oneembodiment. In the example of FIG. 2, the PPG device 100 includes anarrangement of light sources 102 and light detectors 106 similar to thatshown in FIG. 1E. In this embodiment, the PPG device 100 comprises awrist band in which the light sources 102 and light detectors 106 aremounted on or within an underside of the PPG device 100 such that thelight sources 102 and light detectors 106 face the user's skin when wornon the user's wrist. The PPG device 100 may include a fastening means toattach the device to a portion of a user's body. The fastening means maybe a strap that is passed through a receiving portion of the strap andfastened with hook and/or loop fasteners. Other fastening means mayinclude clips, latches, hook-and-loop fasteners such as VELCRO®, clasps,ties, and/or adhesives. The fastening means may be located on any sideof the PPG device 100 such that the fastening device does not interferewith movement or activity.

In an embodiment, the PPG device 100 may comprise a processor, memory,user interface, wireless transceiver, one or more environmental sensors,and one or more biometric sensors other than the detector 106. Forexample, embodiments may be implemented using a monitoring device of thetype shown in U.S. Pat. No. 8,948,832 of Fitbit, Inc., San Francisco,Calif., the entire contents of which are hereby incorporated byreference for all purposes as if fully set forth herein. In other words,the monitoring device of the type shown in U.S. Pat. No. 8,948,832 couldbe modified based upon the additional disclosure herein to result in aworking activity monitoring apparatus capable of performing thefunctions that are described herein. Therefore, the present disclosurepresumes that the reader knows and understands U.S. Pat. No. 8,948,832,and this disclosure is directed to persons having a level of skillsufficient to modify or adapt the monitoring device of the type shown inU.S. Pat. No. 8,948,832 based upon the additional disclosure herein toresult in a working PPG device capable of performing the functions thatare described herein.

Light Sources

In various embodiments, the light sources 102 comprise electronicsemiconductor light sources, such as LEDs, or produce light using any offilaments, phosphors, or laser. In some implementations, each of thelight sources 102 emits light having the same center wavelength orwithin the same wavelength range. In other cases, at least one lightsource 102 may emit light having a center wavelength that is differentfrom another one of the light sources 102. The center wavelengths of thelight emitted by the light sources 102 may be in the range of 495 nm to570 nm. For example, a particular green light source 102 may emit lightwith a center wavelength of 528 nm. In other embodiments, one or more ofthe light sources 102 may emit red light (e.g., 660 nm centerwavelength) or IR light (e.g., 940 nm center wavelength). In someembodiments, one or more of the light sources 102 may emit light withpeak wavelengths typically in the range of 650 nm to 940 nm. Forexample, in various embodiments, a particular red light source may emitlight with a peak wavelength of 660 nm, and an infrared light source mayemit light with peak wavelengths in the range of 750 nm to 1700 nm. Byway of example and not limitation, a particular infrared light sourcemay emit light with a peak wavelength of 730 nm, 760 nm, 850 nm, 870 nm,or 940 nm. In some cases, commercial light sources such as LEDs mayprovide output at about 20 nm intervals with a center wavelengthtolerance of +/−10 nm from the manufacturer's specified wavelength andthus one possible range of useful peak wavelengths for the light sourcesis 650 nm to 950 nm. The green light sources may be configured to emitlight with wavelengths in the range of 495 nm to 570 nm. For example, aparticular green light source may emit light with a wavelength of 528nm. The green light sources may be equally spaced from light detectors106 as the pairs of red and infrared light sources. For example, if thedistance between light detectors 106 and a center of a first red lightsource is 2 mm, the distance between light detectors 106 and a greenlight source may also be 2 mm (e.g., equidistant). In some other cases,the distance between the light detectors and one or more light sourcesis not equidistant. Further, in some embodiments, one or more of thelight sources 102 may comprise a single LED package that emits multiplewavelengths, such as green, red and infrared wavelengths, at the same orsubstantially the same (e.g., less than 1 mm difference) location withrespect to multiple detectors. Such LEDs may include multiplesemiconductor elements co-located using a single die in a singlepackage.

The spacing of the light sources 102 may be measured from the side ofthe light source or the center of the light source. For example, thelight sources may be configured such that the center of each lightsource is at a first distance from the edge of the closest one of thelight detectors 106. In some embodiments, the first distance may be 2mm. In some implementations, each light source is located at a seconddistance from the closest one of the light sources 102, and each lightdetector is located at a third distance from the closest one of thelight detectors 106. In some embodiments, the second and third distancesare identical to the first distance. In other embodiments, each of thesecond and third distances is different from the first distance. Thesecond distance may be identical to or different from the thirddistance. The particular magnitude of the spacing may depend on a numberof factors and this disclosure does not limit the embodiments to anyparticular spacing. For example, spacing in a range of 1 mm (or less) to10 mm would be workable in various embodiments.

In some embodiments, independent control of all light sources isprovided. In other embodiments, several light sources are controlledtogether as a gang or bank. A benefit of independent control of eachlight source, or independent readout from each of multiple detectors(e.g., obtaining independent signals based on the same or differentlight wavelengths from each of multiple detectors), is that a multiplelight path approach may be used to improve the estimation of HR and/orother physiological metrics, as discussed further herein.

Light Detectors

Light detectors 106 comprise one or more sensors that is/are adapted todetect wavelengths of light emitted from light sources 102. A particularlight source 102 combined with a particular detector 106 may comprise asensor such as a PPG sensor. A first PPG sensor and a second PPG sensorcan share components, such as the same light sources and/or detectors,or have different components and thus the term “PPG sensor,” in additionto having its ordinary meaning, may refer to any of such arrangementsalthough actual embodiments may use multiple components in implementinga PPG sensor. The term “PPG device,” in addition to having its ordinarymeaning, may refer to a device including a PPG sensor. A light detector106, in an embodiment, may comprise one or more detectors for detectingeach different wavelength of light that is used by the light sources.For example, a first detector may be configured to detect light with awavelength of 560 nm, a second detector may be configured to detectlight with a wavelength of 940 nm, and a third detector may beconfigured to detect light with a wavelength of 528 nm. Examples includephotodiodes fabricated from semiconductor materials and having opticalfilters that admit only light of a particular wavelength or range ofwavelengths. The light detectors 106 may comprise any of a photodiode,phototransistor, charge-coupled device (CCD), thermopile detector, orcomplementary metal-oxide-semiconductor (CMOS) sensor. The lightdetectors 106 may comprise multiple detector elements, as furtherdescribed herein. One or more of the detectors may comprise a bandpassfilter circuit.

In other embodiments, detector 106 comprises one or more detectorsconfigured to detect multiple wavelengths of light. For example, asingle detector may be configured to tune to different frequencies basedon data received from an electrical digital microprocessor coupled todetectors 106. In another way, the single detector may include multipleactive areas where each active area is sensitive to a given range ofwavelengths. In an embodiment, a single detector is configured to detectlight with wavelengths in the red and IR frequencies and a seconddetector is configured to detect light with wavelengths in the greenfrequencies. Further, each of the light sources 102 may use any of oneor more different wavelengths of light as previously described.

In an embodiment, light detectors 106 are mounted in a housing with oneor more filters that are configured to filter out wavelengths of lightother than wavelengths emitted by light sources 102. For example, aportion of the housing may be covered with a filter which removesambient light other than light in wavelengths emitted by light sources102. For example, signals from light sources 102 may be received at thelight detectors 106 through an ambient light filter that filters out anambient light source that generates an ambient light with a wavelengththat is different from the wavelength that is detected by the detector.Although LEDs and photodiodes are used as examples of the light sources102 and the light detectors 106, respectively, the techniques describedherein may be extended to other types of light sources. For example, thePPG device 100 may include (i) single or multiple LEDs and amulti-element photodetector (e.g., a camera sensor), (ii) an LED arrayand single or multiple photodiodes, (iii) spatial light modulator (SLM)(e.g., a digital micromirror device [DMD] or a liquid crystal on silicon[LCoS] device) and single or multiple LEDs, other combinations thereof,or other configurations of light sources and detectors.

Example Techniques for Generating Heart Rate and Other PhysiologicalMetrics

Certain flow diagrams are presented herein to illustrate various methodsthat may be performed by example embodiments. The flow diagramsillustrate example algorithms that may be programmed, using any suitableprogramming environment or language, to create machine code capable ofexecution by a CPU or microcontroller of the PPG device 100. In otherwords, the flow diagrams, together with the written description in thisdocument, are disclosures of algorithms for aspects of the claimedsubject matter, presented at the same level of detail that is normallyused for communication of this subject matter among skilled persons inthe art to which the disclosure pertains. Various embodiments may becoded using assembly, C, OBJECTIVE-C, C++, JAVA, or other human-readablelanguages and then compiled, assembled, or otherwise transformed intomachine code that can be loaded into ROM, EPROM, or other recordablememory of the activity monitoring apparatus that is coupled to the CPUor microcontroller and then then executed by the CPU or microcontroller.

In an embodiment, PPG signals obtained from multiple light paths may beprocessed to filter or reject signal components that are associated withmotion of the user, using a computer program to identify the motioncomponent of the signal and remove the identified motion component fromthe composite signal, leaving the cardiac component as a remainder orfinal signal.

In various embodiments, the methods of FIGS. 3-7 may be performed by oneor more of: firmware operating on the PPG device or a secondary device,such as a mobile device paired to the PPG device, a server, hostcomputer 130, and the like. For example, the PPG device may executeoperations relating to generating the PPG signals which are uploaded orotherwise communicated to a server that performs operations for removingthe motion components and creating a final estimate value for HR, SpO2,and/or other physiological metrics. Alternatively, the PPG device mayexecute operations relating to generating the PPG signals and removingthe motion components to produce a final estimate value for HR, SpO2,and/or other physiological metrics local to the PPG device 100. In thiscase, the final estimate may be uploaded or otherwise communicated to aserver such as host computer 130 that performs other operations usingthe value.

Generating Estimate for Physiological Metric Based on MultipleSource-Detector Pairs

FIG. 3 illustrates an example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure. The method 300 may be operable by a PPG device 100, orcomponent(s) thereof, for generating HR or other physiological metricsin accordance with aspects of this disclosure. For example, the steps ofmethod 300 illustrated in FIG. 3 may be performed by a processor 110 ofthe PPG device 100. In another example, a user device (e.g., mobilephone) or a server in communication with the PPG device 100 may performat least some of the steps of the method 300. For convenience, themethod 300 is described as being performed by the processor 110 of thePPG device 100.

At block 305, the processor 110 obtains one or more first PPG signalsbased on light received by a first light detector from a set of lightsources. Each light source in the set may have a spatial location thatis different from that of another light source in the set with respectto the first light detector. Further, each light source in the set maybe configured to emit light according to an emission schedule that isdifferent from that of another light source in the set. For example, afirst light source in the set may emit light for a given time period ateven seconds (e.g., 2, 4, 6, etc. 2/4, 4/4, 6/4, etc., or 2/8, 4/8, 6/8,etc., to name a few examples), and a second light source in the set mayemit light for a given time period at odd seconds (e.g., 1, 3, 5, etc.1/4, 3/4, 5/4, etc., or 1/8, 3/8, 5/8, etc., to name a few examples).The first PPG signals may be obtained from the one or more lightdetectors 106 based upon light sources operating at any of thefrequencies that have been described above. For example, by executinginstructions stored in memory 112 of FIG. 1B as control program 118, theprocessor 110 may signal the driver 122 to activate one, some, or all ofthe light sources 102, which produce light directed toward bloodvessels, which is then reflected to the first light detector. The firstlight detector produces a responsive signal to the bus 116 that is sentto the processor 110 and then stored in a register of the processor 110or in the memory 112. The first PPG signals may be represented in anysuitable form capable of being processed by a processor, such as theanalog signals or digital data sampled from the analog components andstored in computer memory. In an example, the first PPG signals maycorrespond to green light previously emitted by a light source (or lightsources) after the emitted light has interacted with a user's skin, whenthe PPG device 100 is worn. The first PPG signals may include a motioncomponent and a cardiac component. In another example, the first PPGsignals may include a motion component and another physiologicalcomponent.

At block 310, the processor 110 obtains one or more second PPG signalsbased on light received by a second light detector from the set of lightsources, each light source in the set having a spatial location that isdifferent from another light source in the set with respect to thesecond light detector. For example, in a manner similar to thatdescribed above for block 302, the processor 110 drives the set of lightsources 102 and obtains second PPG signals from the second lightdetector different from the first light detector discussed at block 305.Again, the second set of PPG signals may be represented in any suitableform capable of being processed by a processor, such as the analogsignals or digital data sampled from the analog components and stored incomputer memory. In an example, the second PPG signals may correspond togreen light previously emitted by a light source (or light sources)after the emitted light has interacted with a user's skin, when the PPGdevice 100 is worn. The second PPG signals may include a motioncomponent and a cardiac component. In another example, the first PPGsignals may include a motion component and another physiologicalcomponent. In some embodiments, the light sources that are used toobtain the first and second PPG signals may use light having the samewavelength.

At block 315, the processor 110 generates a physiological metric basedon one or more of the first and second PPG signals. Various techniquesfor generating the HR, SpO2, and/or other physiological metric value maybe used. For example, template matching based on a template previouslycreated and stored from a high-quality PPG signal can be used to providefilter coefficients for a matched filter to maximize the signal-to-noiseratio, or an adaptive filter may be used that tunes a band-pass inreal-time based upon heartbeat data derived from the PPG datasets,usually based upon green light sources. Once an HR estimate or othermetrics has/have been generated, signals, data values, or data pointsthat are available or apparent in the second PPG signals may be used tomodify or improve the first PPG signals so that the resulting modifiedor improved first PPG signals can be transformed into a more accurate HRestimate. For example, if the intensity of the motion artifact signalvaries in space between two or more light paths (i.e. source-detectorpairs), a mathematical model can be applied to this spatial intensitydifference to remove the motion artifact signal, thus isolating thesignal of interest (e.g. cardiac or HR signal).

At block 320, the processor 110 causes the physiological metric to bedisplayed via a user interface on a user device. For example, theprocessor 110 may drive the display 114 to display the generated HR,SpO2, and/or other physiological metric value. In other embodiments, themethod 300 may be programmed to cause uploading the estimated HR, SpO2,and/or other physiological metric value to host computer 130 for furtherprocessing, display, or reporting.

In the method 300, one or more of the blocks shown in FIG. 3 may beremoved (e.g., not performed) and/or the order in which the method 300is performed may be switched. For example, in some embodiments, block320 may be omitted. In some embodiments, additional blocks may be addedto the method 300. The embodiments of the present disclosure are notlimited to or by the example shown in FIG. 3, and other variations maybe implemented without departing from the spirit of this disclosure.

Time Multiplexing

In some embodiments, a single light detector 106 receives reflectedlight originating from multiple light sources 102. The light detector106 may distinguish one PPG signal based on light emitted by lightsource A from another PPG signal based on light emitted by light sourceB by using time multiplexing (e.g., at block 305 to distinguish amongthe first PPG signals, and at block 310 to distinguish among the secondPPG signals). In some embodiments, light source A emits light accordingto a first emission schedule that is different from a second emissionschedule of light source B. The first emission schedule may specify thatlight source A is to emit light for X seconds every Y seconds, and thesecond emission schedule may specify that light source B is to emitlight for P seconds every Q seconds. In some cases, X may equal A, and Ymay equal B. In other cases, each of X, Y, P, and Q may have the samevalue. In some cases, the period during which light source A emits lightmay differ from the period during which light source B emits light by anoffset. For example, the PPG device 100 may activate (e.g., emit lightvia) light source A during a temporal window T1 (without emitting anylight via light source B during T1) and activate light source B during atemporal window T2 (without emitting any light via light source A duringT2). The light emitted by the light source A during T1 may be detectedby the light detector during a temporal window T1′ and the light emittedby the light source B during T2 may be detected by the light detectorduring a temporal window T2′. Thus, the PPG device 100 can determinefirst PPG signals originating from light source A based on lightdetected during T1′ and determine second PPG signals originating fromlight source B based on light detected during T2′. In some embodiments,there is no temporal gap between T1 and T2 (and/or T1′ and T2′). Inother embodiments, there is a temporal gap between T1 and T2 (and/or T1′and T2′). In some embodiments, there is a partial overlap between T1 andT2.

In some embodiments, at most one light source is emitting light at anygiven moment. In some embodiments, as long as any light source isemitting light, all of the light detectors are sensing reflected lightfrom the activated light source(s). In some implementations, an internalclock is maintained, and the processor 110 may identify the specificlight path (e.g., the light source and the light detector) based on theinternal clock and a light source activation schedule indicative ofwhich light sources are emitting during which temporal windows.

In some implementations, a single amplifier is used to amplify thesignals generated by multiple light detectors 106, and the use of themultiple light detectors 106 may also be time-multiplexed. For example,the amplifier may amplify signals from light detector A during T0-T1,amplify signals from light detector B during T1-T2 (or T2-T3 with theamplifier not amplifying any signals during T1-T2), and so on. In othercases, each light detector 106 may be associated with a separateamplifier and multiple amplifiers may amplify signals generated bymultiple light detectors 106 during a given time period. Although timemultiplexing is described herein as a method of distinguishing betweendifferent PPG signals, other multiplexing techniques such as frequencymultiplexing and wavelength multiplexing may be used to distinguish thePPG signals.

Spatially Resolving Measurements

The PPG device 100 may utilize the spatial location associated with eachof the light sources 102 and the light detector 106 to further improvethe estimate for HR, SpO2, and/or other physiological metric. In someembodiments, the PPG device 100 compares the PPG signals determinedbased on two (or more) unique light paths and identifies any discrepancyamong the PPG signals. For example, if the phases of the two PPG signalsare different, the PPG device 100 may discard one of the signals (e.g.,based on the signal quality). In some cases, the PPG device 100 maydetermine the cause of the discrepancy and take different actions basedon the cause. For example, if the PPG signals indicate that thediscrepancy may be caused by the movement of the PPG device 100, the PPGdevice 100 may average two or more of the PPG signals and generate theestimate for HR, SpO2, and/or other physiological metric based on theaveraged signal(s). On the other hand, if the PPG signals indicate thatthe discrepancy may be caused by the placement of the sensors or aforeign object in the light path (e.g., hair or other objects), the PPGdevice 100 may discard the affected PPG signals and generate theestimate for HR, SpO2, and/or other physiological metric based on theremaining PPG signal(s). For example, if the PPG device 100 determinesthat light paths involving light sources located on one side of the PPGdevice 100 are underperforming, the PPG device 100 may generate theestimate for HR, SpO2, and/or other physiological metric based on lightpaths involving light sources on the opposite side of the PPG device100.

In some embodiments, the estimation for HR, SpO2, and/or otherphysiological metric may be agnostic to the spatial arrangement of thelight sensors and detectors. For example, the PPG device 100 maycalculate a spatially-agnostic metric for each PPG signal and generatethe estimate for HR, SpO2, and/or other physiological metric based onthe PPG signal having the most desirable metric value.

Generating Estimate for Physiological Metric based on MultipleSource-Detector Pairs

FIG. 4 illustrates an example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure. The method 400 may be operable by a PPG device 100, orcomponent(s) thereof, for generating HR or other physiological metricsin accordance with aspects of this disclosure. For example, the steps ofmethod 400 illustrated in FIG. 4 may be performed by a processor 110 ofthe PPG device 100. In another example, a user device (e.g., mobilephone) or a server in communication with the PPG device 100 may performat least some of the steps of the method 400. For convenience, themethod 400 is described as being performed by the processor 110 of thePPG device 100.

At block 405, the processor 110 obtains one or more first PPG signalsbased on light received by a first light detector from a set of lightsources, each light source in the set having a spatial location that isdifferent from another light source in the set with respect to the firstlight detector. The processor 110 may obtain the first PPG signals in amanner similar to that described with reference to block 305.

At block 410, the processor 110 obtains one or more second PPG signalsbased on light received by a second light detector from the set of lightsources, each light source in the set having a spatial location that isdifferent from another light source in the set with respect to thesecond light detector. The processor 110 may obtain the second PPGsignals in a manner similar to that described with reference to block310.

At block 415, the processor 110 extracts a physiological component fromat least one of the first and second PPG signals by comparing the firstand second PPG signals. For example, the physiological component may bea cardiac component. In another example, the physiological component maybe a component other than a cardiac component and usable to generate aphysiological metric. As discussed herein, the processor 110 may comparethe first and second PPG signals to identify the motion component. Forexample, light paths involving light sources having different spatiallocations may have different motion characteristics. For example,signals from a first light path may be known to be more sensitive toblood flow (or a factor other than motion) or otherwise have arelatively higher cardiac component (or another physiological component)in comparison to a motion component, while signals from a second lightpath may be more sensitive to motion or otherwise have a relativelyhigher motion component in comparison to a cardiac component (or anotherphysiological component). With such data, the method may use the firstlight path to sense the cardiac component (or another physiologicalcomponent) of a signal and the second, different light path to sense themotion component. As a non-limiting example, the second light path maybe one where the light detector is further away from the light source incomparison to the first light path.

In some embodiments, if the processor 110 determines that the majorityof a PPG signal (e.g., one of the first and second PPG signals) is acardiac component (or another physiological component), the processor110 may determine the estimate for HR, SpO2, and/or other physiologicalmetric based on the PPG signal. If the processor 110 determines that themajority of a PPG signal (e.g., one of the first and second PPG signals)is a motion component, the processor 110 may subtract the motioncomponent from the signal (e.g., motion component determined based onaccelerometer readings) (or otherwise modify the PPG signal to reducethe effect of the motion component) and determine the estimate for HR,SpO2, and/or other physiological metric based on the PPG signal withoutthe motion component. In some embodiments, for the purpose ofdetermining an HR or a cardiac component, any PPG signal outside therange of a cardiac signal may be discarded. For example, if the PPGdevice 100 determines that a PPG signal would yield an HR (or BPM)estimate of 300, the PPG device 100 may discard the PPG signal. In someimplementations, if the PPG device 100 that the PPG device 100 isundergoing a periodic motion (e.g., based on activity detection or basedon sensor data), the PPG device 100 may identify the motion componentcorresponding to the periodic motion and remove the motion componentfrom the PPG signals. Techniques for motion component removal (e.g.,motion artifact removal) are discussed in U.S. Pat. No. 9,005,129 ofFitbit, Inc., San Francisco, Calif., the entire contents of which arehereby incorporated by reference for all purposes as if fully set forthherein. Other techniques that can be used include, but are not limitedto, ICA and other forms of blind source separation.

At block 420, the processor 110 generates a physiological metric basedon the physiological component. For example, the processor 110 may drivethe display 114 to display the generated value for HR, SpO2, and/orother physiological metric. In other embodiments, the method 400 may beprogrammed to cause uploading the estimated value for HR, SpO2, and/orother physiological metric to host computer 130 for further processing,display, or reporting.

In the method 400, one or more of the blocks shown in FIG. 4 may beremoved (e.g., not performed) and/or the order in which the method isperformed may be switched. In some embodiments, additional blocks may beadded to the method 400. The embodiments of the present disclosure arenot limited to or by the example shown in FIG. 4, and other variationsmay be implemented without departing from the spirit of this disclosure.

Measuring Heart Rate Based on PPG and Motion Data

FIG. 5 is an example block diagram of a system used for determining HRin accordance with aspects of this disclosure. As shown in FIG. 5, thePPG device 100 may include a system 500 of circuit components fordetermining the HR of the user based on an optical PPG signal (e.g.,received by one or more light detectors 106 of the PPG device 100) and amotion signature (e.g., received from an accelerometer in the PPG device100). As used herein, a motion signature may refer to any biometricsignature or signal that may be received from and/or based on outputdata from one or more of sensors, such as, for example, inertialsensor(s) (e.g., accelerometer(s) and gyroscope(s)), barometricsensors(s) (e.g., altimeter(s)), which may be indicative of the activityand/or physiological state of a user of the PPG device 100. The system500 may be implemented by hardware components and/or in softwareexecuted by the processor 110. The system 500 may include first andsecond spectra estimators 501 and 502, a multi-spectra tracker 503, anactivity identifier or discriminator 504, and a track selector 505. Eachof the first and second spectra estimators 501 and 502 may include aFast Fourier Transform (FFT) block and a peak extraction block. In theexample of FIG. 5, the activity identifier 504 may use the peaksextracted from the motion signature to determine the activity that theuser is performing (e.g., sedentary, walking, running, sleeping, lyingdown, sitting, biking, typing, elliptical, weight training, swimming,etc.). This determination of the current activity of the user may beused by the multi-spectra tracker 503 and the track selector 505 inextracting the HR from the optical PPG signal. Thus, the motionsignature in FIG. 5 may be used by the system 500 to determine thecurrent activity of the user. In other embodiments, the processor 110may use a technique similar to that of the activity identifier 504 indetermining the type of an exercise, as discussed in greater detailbelow.

The blocks illustrated in FIG. 5 are merely examples of componentsand/or processing modules that may be performed to supplement a PPGsignal with a motion signature to determine HR. However, in otherimplementations, the system 500 may include other blocks or may includeinput from other biometric sensors of the PPG device 100.

Under certain operating conditions, the HR of the user may be measuredby counting the number of signal peaks within a time window or byutilizing the fundamental frequency or harmonic frequency components ofthe signal (e.g., via an FFT). In other cases, such as HR data acquiredwhile the user is in motion, FFTs may be performed on the signal andspectral peaks extracted, which may then be subsequently processed by amultiple-target tracker which starts, continues, merges, and/or deletestracks of the spectra.

In some embodiments, a similar set of operations may be performed on themotion signature and the output may be used to perform activitydiscrimination which may be used to assist the multi-spectra tracker503. For instance, it may be determined that the user was stationary andhas begun to move. This information may be used to by the multi-spectratracker 503 to bias the track continuation toward increasingfrequencies. Similarly, the activity identifier 504 may determine thatthe user has stopped running or is running slower and this informationmay be used to preferentially bias the track continuation towarddecreasing frequencies.

Tracking may be performed by the multi-spectra tracker 503 withsingle-scan or multi-scan, multiple-target tracker topologies such asjoint probabilistic data association trackers, multiple-hypothesistracking, nearest neighbor, etc. Estimation and prediction in thetracker may be done through Kalman filters, spline regression, particlefilters, interacting multiple model filters, etc.

The track selector 505 may use the output tracks from themultiple-spectra tracker 503 and estimate the user's HR based on theoutput tracks. The track selector 505 may estimate a probability foreach of the tracks that the corresponding track is representative of theuser's HR. The estimate may be taken as the track having the maximumprobability of being representative of the user's HR, a sum of thetracks respectively weighted by their probabilities of beingrepresentative of the user's the HR, etc. The activity identifier 504may determine a current activity being performed by the user which maybe used by the track selector 505 in estimating the user's HR. Forinstance, when the user is sleeping, sitting, lying down, or sedentary,the user's estimated HR may be skewed toward HRs in the 40-80 BPM range.When the user is running, jogging, or doing other vigorous exercise, theuser's estimated HR may be skewed toward elevated HRs, such as, forexample, in the 90-180 BPM range. The activity identifier 504 maydetermine the user's current activity (e.g., a current exercise) basedat least in part on the speed of the user. The user's estimated HR maybe shifted toward (or wholly obtained by) the fundamental frequency ofthe selected output track when the user is not moving. The output trackthat corresponds to the user's HR may be selected by the track selector505 based on criteria that are indicative of changes in activity. Forinstance, when the user begins to walk from being stationary, the trackselector 505 may select the output track that illustrates a shift towardhigher frequency based on output received from the activitydiscriminator 504.

Although some embodiments of the present disclosure are described withrespect to HR, the techniques described herein may be extended to othermetrics. For example, sensor data generated by the one or more sensorsdescribed herein may be used to determine respiration, SpO2, bloodvolume, blood glucose, skin moisture, and skin pigmentation level and,for example, utilize such metrics for activity detection/identification.Further, the motion removal techniques described in the presentdisclosure may be used in conjunction with other motion removaltechniques.

Removing Motion Component from PPG Signals

FIG. 6 illustrates an example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure. The method 600 may be operable by a PPG device 100, orcomponent(s) thereof, for generating HR or other physiological metricsin accordance with aspects of this disclosure. For example, the steps ofmethod 600 illustrated in FIG. 6 may be performed by a processor 110 ofthe PPG device 100. In another example, a user device (e.g., mobilephone) or a server in communication with the PPG device 100 may performat least some of the steps of the method 600. For convenience, themethod 600 is described as being performed by the processor 110 of thePPG device 100.

At block 605A, the processor 110 acquires signal A from path A. At block605B, the processor 110 acquires signal B from path B. In an embodimenthaving N unique paths, at block 605N, the processor 110 acquires signalN from path N, where N is a natural number. Although not shown in FIG.6, there may be any number of similar acquisition steps between blocks605B and 605N for each unique path.

At block 610, the processor 110 fits the data to a model. For example,the processor 110 may determine whether the PPG signals can be fitted toone of the mathematical models stored on the PPG device 100 to isolatethe cardiac component (or any other physiological component) from themotion component.

At block 615, the processor 110 applies a linear transform. For example,the processor 110 may apply a linear transform to the motion componentidentified at block 610 before subtracting the motion component from thecomposite PPG signals (e.g., signals including both cardiac and motioncomponents, or signals including the motion component and anotherphysiological component). Other correction techniques may include ICAand other forms of blind source separation.

At block 620, the processor 110 estimates a BPM. For example, theprocessor 110 may estimate the BPM based on the resulting PPG signalfrom which the motion component has been removed at block 615.

In the method 600, one or more of the blocks shown in FIG. 6 may beremoved (e.g., not performed) and/or the order in which the method isperformed may be switched. In some embodiments, additional blocks may beadded to the method 600. The embodiments of the present disclosure arenot limited to or by the example shown in FIG. 6, and other variationsmay be implemented without departing from the spirit of this disclosure.

Selecting Best PPG Signal

FIG. 7 illustrates an example method of generating a physiologicalmetric using multiple light paths in accordance with aspects of thisdisclosure. The method 700 may be operable by a PPG device 100, orcomponent(s) thereof, for generating HR or other physiological metricsin accordance with aspects of this disclosure. For example, the steps ofmethod 700 illustrated in FIG. 7 may be performed by a processor 110 ofthe PPG device 100. In another example, a user device (e.g., mobilephone) or a server in communication with the PPG device 100 may performat least some of the steps of the method 700. For convenience, themethod 700 is described as being performed by the processor 110 of thePPG device 100.

At block 705A, the processor 110 acquires signal A from path A. At block705B, the processor 110 acquires signal B from path B. In an embodimenthaving N unique paths, at block 705N, the processor 110 acquires signalN from path N, where N is a natural number. Although not shown in FIG.7, there may be any number of similar acquisition steps between blocks705B and 705N for each unique path.

At block 710A, the processor 110 estimates a BPM based on signal A. Atblock 710B, the processor 110 estimates a BPM based on signal B. In anembodiment having N unique paths, at block 710N, the processor 110estimates a BPM based on signal N, where N is a natural number. Althoughnot shown in FIG. 7, there may be any number of similar estimation stepsbetween blocks 710B and 710N for each unique path.

At block 715, the processor 110 selects the best BPM estimate using aconfidence metric. For example, the processor 110 compares the PPGsignals corresponding to multiple light paths using a confidence/qualitymetric such as SNR and selects the PPG signal having the highestconfidence/quality to be used for estimating the HR of the user. In oneembodiment, the processor 110 selects one estimate, among those obtainedvia blocks 710A-710N, that has the highest confidence metric (e.g.,SNR). In some embodiments, the confidence/quality metric is based on thecharacteristics of the waveform (e.g., waveform feature, waveformfidelity, etc.). In an embodiment, the method 700 may use hysteresislogic that prevents jumping between PPG signals of two different lightpaths within a short time window, for example, if the confidence valuesof both are within a specified tolerance value.

In another embodiment, the processor 110 selects a BPM estimate based onan activity-specific confidence metric. For example, if the user isdetermined to be running, the processor 110 may calculate arunning-specific confidence metric for each BPM estimate and select aBPM estimate based on the individual running-specific confidence metricvalues. As another example, if the user is determined to be sitting, theprocessor 110 may calculate a sitting-specific confidence metric foreach BPM estimate and select a BPM estimate based on the individualsitting-specific confidence metric values. In yet another example, ifthe user is determined to be sleeping, the processor 110 may calculate asleeping-specific confidence metric for each BPM estimate and select aBPM estimate based on the individual sleeping-specific confidence metricvalues. The processor 110 may perform activity detection according toany known activity detection techniques.

At block 720, the processor 110 applies a smoothing filter. For example,the processor 110 may apply the smoothing filter to smooth the estimatedHR value. Filtering can be performed to improve accuracy, or to presenta better user experience, or for both.

In the method 700, one or more of the blocks shown in FIG. 7 may beremoved (e.g., not performed) and/or the order in which the method isperformed may be switched. In some embodiments, additional blocks may beadded to the method 700. The embodiments of the present disclosure arenot limited to or by the example shown in FIG. 7, and other variationsmay be implemented without departing from the spirit of this disclosure.

Physiological Metrics Based on Machine Learning

In some embodiments, the processor 110 processes multiple PPG signals.For example, each PPG signal of the multiple PPG signals may represent aunique source-detector pair (e.g., light source A to light detector X).The signal quality of the multiple PPG signals may vary depending on avariety of factors (time, location on the user's wrist, weather,orientation of the device, how tightly the user is wearing the device,magnitude and/or direction of movement experienced by the device,activity being performed by the user, and the like). Thus, the processor110 may determine which one or ones of the multiple PPG signals may moreaccurately represent the various physiological metrics of the user suchthat the resulting value is more accurate than that resulting fromrelying on a single source-detector pair regardless of the changes inthe various factors discussed above.

In some embodiments, machine learning can be used to update or optimizethe way that the processor 110 processes the multiple PPG signals togenerate a value for the physiological parameters. For example, theprocessor 110 may access a set of PPG signals and determine, using asignal analysis engine stored in one or more memories of the wearabledevice (e.g., memory 112), whether the accessed set of PPG signalsrequires updating weight values corresponding to the individual signalsin the set. The signal analysis engine can be trained, usingunsupervised learning (e.g., in some cases, further based on motion datafrom the accelerometer and/or previously done supervised learning), toupdate the weight values based on one or more metrics (e.g., confidencevalues) associated with the individual signals in the set or otherenvironmental factors (e.g., time, location on the user's wrist,weather, orientation of the device, how tightly the user is wearing thedevice, magnitude and/or direction of movement experienced by thedevice, activity being performed by the user, and the like). Forexample, the processor 110 may determine the final value to be outputtedon the graphical user interface based on the PPG signal (or an estimatedvalue based on the PPG signal) with the highest confidence value. Insome cases, some or all of the PPG signals (e.g., two or more PPGsignals with the highest confidence values, or two or more estimateswith the highest confidence values) may be averaged or weighted for thepurpose of determining the final value.

The processor 110 may be notified (e.g., by a motion sensor of thewearable device) of movements undergone by the wearable device. Forexample, the motion sensor may be an accelerometer configured togenerate motion data indicative of the movements undergone by thewearable device. The processor 110 may output a final value of thephysiological metric tracked by the wearable device based at least onthe updated weight values. The final value may be displayed on a displayof the wearable device.

In some embodiments, the confidence values (associated with theindividual PPG signals or estimates generated therefrom), the finalvalues of the physiological metrics, the identity of the PPG signals (orsource-detector paths) used for generating the final values, and/or theweights corresponding to the individual PPG signals (or source-detectorpaths), or any combination thereof, can serve as a training data set forupdating or improving (e.g., by adjusting the individual weights) theway in which the processor 110 determines the final values of thephysiological metrics. For example, if the confidence values associatedwith a given source-detector path exhibit a pattern of being lower thana threshold value when there is user motion along a given rotationalaxis, the weight value associated with given source-detector path may bereduced when the processor 110 detects user motion along the givenrotational axis. As another example, if the confidence values associatedwith a given source-detector path exhibit a pattern of being higher thana threshold value regardless of environmental conditions, the weightvalue associated with given source-detector path may be increased forall environmental conditions.

Combining PPG Signals

In some embodiments, at least some of the PPG signals are averagedtogether to produce a single PPG signal. For example, two of the threePPG signals corresponding to the three unique light paths are averagedtogether to produce a single PPG signal. Then, the PPG device 100 maygenerate HR, SpO2, and/or other physiological metrics based on theaveraged PPG signal and the remaining one of the three PPG signal. Forexample, in some cases, the light detector can be a camera sensorconsisting of millions of pixels or individual detectors. Sinceperforming a BPM estimation on millions of individual pixels may bedifficult, the PPG device 100 may combine the signals received at someof the pixel locations by averaging such pixel signals together, andperforming the BPM estimation on the fewer number of signals. In someimplementations, even with the averaging of at least some of the PPGsignals, the number of signals based on which the PPG device 100generates the HR, SpO2, and/or other physiological metrics is greaterthan 1. In other embodiments, none of the PPG signals are averagedtogether (i.e., each PPG signal is separately considered by the PPGdevice 100).

Adaptive Activation of Sensors

Additionally or alternatively, the processes of FIGS. 3, 4, 6, and 7 mayutilize input from biometric sensors or environmental sensors indicatingan ambient temperature or a skin temperature of the wearer of theactivity monitoring apparatus. Based on changes in temperature valuesreceived from these sensors, the process may energize or de-energizedifferent arrangements of light sources 102. In this embodiment, theprocesses of FIGS. 3, 4, 6, and 7 may further include: activating only afirst set of light sources and deactivating all other light sources;receiving, at the one or more light detectors of the PPG device, firstPPG signals from only the first set of light sources; receiving anambient temperature signal from an ambient temperature sensor that islocated proximate to the first set of light sources; determining thatthe ambient temperature signal is less than a stored minimum temperaturesignal; based on the determination, activating a second set of lightsources; receiving, at the one or more light detectors of the PPGdevice, second PPG signals from the second set of light sources;combining at least some of the first and second PPG signals; andgenerating HR, SpO2, and/or other physiological metrics based on thecombined signals. Although the ambient temperature is used as anexample, the selective activation of light sources can be extended toother sensor data. Further, light detectors can also be selectivelyactivated based on sensor data obtained by the PPG device.

In some embodiments, one or more light paths are selectively activatedbased on the user activity. For example, if it is determined that theuser is currently sleeping, only a low number of light paths may beactivated (e.g., 1 or 2), if it is determined that the user is currentlywalking, a higher number of light paths may be activated (e.g., 3 or 4),and if it is determined that the user is currently engaging in a highintensity exercise (e.g. running), an even higher number of light pathsmay be activated (e.g., 5 or 6). Alternatively, in some embodiments,different activities may result in activation of different but the samenumber of light paths. For example, if it is determined that the user iscurrently sleeping, only light paths A and B may be activated, if it isdetermined that the user is currently walking, only light paths C and Dmay be activated, and if it is determined that the user is currentlyengaging in an exercise, only light paths E and F may be activated. Inother embodiments, the number and the identity of light paths may bothvary depending on the detected activity. For example, if it isdetermined that the user is currently sleeping, only light path A may beactivated, if it is determined that the user is currently walking, onlylight paths B and C may be activated, and if it is determined that theuser is currently engaging in an exercise, only light paths D, E, and Fmay be activated. Such embodiments allow activity-specific arrangementof light sources and detectors to be used, such that the signal-to-noiseratio is improved or optimized. Although described in the activitydetection context, these techniques can be extended to movementdetection to provide movement-state-specific (e.g., based on whether thedevice is undergoing no movement, movement less than a threshold, movinggreater than a threshold value, etc.) arrangement of light sources anddetectors.

As discussed above, one of the advantages in some of the embodimentsdescribed herein is that the spatial information associated with thelight sources and/or light detectors can be used by different algorithmsto improve estimation accuracy of the PPG sensing device for HR, SpO2,and/or other physiological metrics, especially when the user of thedevice is exercising or performing activities involving motion. Existingimplementations typically rely on algorithms to improve the estimationperformance for HR, SpO2, and/or other physiological metrics, but do nothave the benefit of the extra sensor data generated based on multiplelight paths.

The present disclosure is also directed to systems and methods forgenerating a heart rate prediction as a probability distribution. Forexample, in an embodiment, systems and methods of the present disclosureperform machine learning techniques to generate an output of multipleheart rate estimates with their corresponding probabilities. Forexample, in one embodiment, the output may include one or morehistograms. Such techniques can be further understood with reference toFIGS. 8-14 as described herein below.

Referring now to FIG. 8, a flow chart of one embodiment of a method 800for generating a heart rate estimation for a user device in accordancewith aspects of this disclosure is illustrated. In the method 800, oneor more of the blocks shown in FIG. 8 may be removed (e.g., notperformed) and/or the order in which the method 800 is performed may beswitched. In some embodiments, additional blocks may be added to themethod 800. The embodiments of the present disclosure are not limited toor by the example shown in FIG. 8, and other variations may beimplemented without departing from the spirit of this disclosure.

As shown at 802, the method 800 includes obtaining multiple PPG signalsbased on light received by at least one light detector of the userdevice from multiple light sources during different temporal windowsduring which other light sources of the multiple light sources are notemitting light. Further, the multiple PPG signals each represent adifferent channel of light (also referred to herein as a light path)received by the light detector(s) from the multiple light sources. Forexample, as shown in FIG. 9, a schematic diagram of one embodiment of anarrangement of the light sources S1, S2 and the light detectors D1, D2of the wearable PPG device 100 is illustrated. In particular, as shown,the PPG device 100 includes two light sources S1, S2 and two lightdetectors D1, D2. Thus, FIG. 9 is similar to the embodiment of FIG. 1G,which also includes two light sources 102 and two light detectors 106.Accordingly, as shown, the PPG device 100 includes a total of fourchannels as indicated by reference numbers 1, 2, 3, and 4 (i.e., whichrefers to a single, distinct light path from a particular light sourceto a particular light detector).

It should be understood by those having ordinary skill in the art thatthough four channels are provided in the illustrated embodiment, anysuitable number of channels may be used as inputs, including more thanfour channels and fewer than four channels. With a single channel, thepreferred approach is generally to select the heart rate with thehighest probability, and then use the selected heart rate for thecurrent heart rate. With multiple channels, however, the method 800 ofthe present disclosure is capable of more accurately estimating heartrate by considering multiple heart rates with varying probabilities. Forexample, in one embodiment, the method 800 may include summing theprobability distributions from multiple PPG channels and using themaximum of the sum to determine the heart rate, as further describedherein. If the resulting combined probability distribution has a largespread, this can be used as an indicator of low quality signal, whichmay be used as a confidence metric. In an embodiment, for example, thespread of the resulting probability distribution is the variance of thedistribution.

For example, referring back to FIG. 8, as shown at 804, the method 800includes processing the multiple PPG signals to produce an outputcorresponding to a heart rate probability distribution for the multiplePPG signals. More specifically, in an embodiment, the method 800 mayinclude processing the multiple PPG signals to produce the outputcorresponding to the heart rate probability distribution for themultiple PPG signals via a trained machine learning model of the userdevice.

For example, as shown in FIG. 10, the PPG device 100 may be configuredwith a computing system 1000 that receives various sensor inputs 1002(e.g., from the light detectors D1, D2 and/or an accelerometer (i.e.,from accel) of the PPG device 100) and processes such inputs 1002 toproduce an output 1008 (i.e., heart rate HR).

In addition, as shown at 1004, the computing system 1000 may also beconfigured to pre-process the inputs 1002. In an embodiment, forexample, the computing system 100 may select a certain time period fromthe channels and accelerometer signals (such as the last 8 seconds, 10seconds, or any other suitable time frame). The computing system 1000may subsequently subtract the means from these signals and divide by thestandard deviations of the signals. Moreover, in an embodiment, thecomputing system 1000 can multiply the signals by a Hamming window andapply a fast Fourier transform (FFT). In addition, in an embodiment, thecomputing system 1000 can down-select the Fourier frequency bins ofinterest in a realistic range of heart rate values, for example, 0.5Hertz (Hz) (30 bpm) to 4 Hz (240 bpm). Thus, the magnitude and/or phaseof the Fourier transform can be used by the computing system 1000 as aninput to a trained machine learning model 1006. In alternativeembodiments, the PPG signals may be fed directly to the machine learningmodel 1006 without any pre-processing.

Furthermore in an embodiment, the trained machine learning model 1006may include a neural network. Moreover, in an embodiment, as shown, themachine learning model 1004 may also receive historical data 1010relating to the PPG device 100, such as PPG signal data and/or data fromthe accelerometer(s). As used herein, a neural network generally refersto an artificial neural network that is based on a collection ofconnected units or nodes called artificial neurons.

In particular, an example neural network 1100 is illustrated in FIG. 11.As shown, the neural network 1100 includes an interconnected group ofnodes 1102. Further, as shown, the circular nodes 1102 represent anartificial neuron and the arrows 1104 represent a connection from theoutput of one artificial neuron to the input of another. Thus, eachconnection can transmit a signal to other nodes 1102. In other words,the nodes 1102 are configured to receive a signal, process the signal,and then signal other nodes connected to it. In an embodiment, thesignal at a connection is a real number, and the output of each neuronis computed by some non-linear function of the sum of its inputs.Further, as shown, the neurons/nodes 1102 are aggregated into layers1106 that may perform different transformations on their inputs toarrive at a desired output 1108. In particular, as shown, the layers1106 of the illustrated neural network 110 include an input layer, afirst hidden layer, a second hidden layer, and an output layer. Itshould be understood by those having ordinary skill in the art that anysuitable number of layers 1106 may be included in the neural network1100 to achieve the desired output 1108. In addition, as shown, theoutputs 1108 in the illustrated neural network 1100 correspond to theheart rate probability distribution (e.g., [78, p(78)], [79, p(79)],[80, p(80)], [81, p(81)], [82, p(82)], and so on) for the multiple PPGsignals, where “78”, as an example, represents the heart rate and p(78)represents the probability of this heart rate.

Thus, in an embodiment, the method 800 may include determining aconfidence metric or probability associated with each of the multiplePPG signals. In such embodiments, the confidence metric is indicative ofa signal quality for a particular light path associated with each of themultiple PPG signals. In addition, in an embodiment, the method 800 mayinclude determining a sum of probabilities for all heart rates within afixed number of beats-per-minute of the heart rate estimation and usingthe sum as the confidence metric.

Moreover, and referring now to FIG. 12, an example neural networkarchitecture 1200 is illustrated with sample inputs and outputs for eachof the layers 1202. More specifically, as shown, the example neuralnetwork architecture 1200 is an artificial neural network, or moreprecisely, a convolutional neural network (CNN). Furthermore, the neuralnetwork architecture 1200 includes six layers 1202, namely, input_1:InputLayer, conv2D: Conv2D, conv2D_1: Conv2D, flatten: Flatten, dense:Dense, and dense_1: Dense. Thus, as generally understood, theillustrated neural network architecture 1200 includes a stack of layers1202 which are defined by the action of a number of filters on theinput. In particular, as shown, the input (58, 2, 2) for the input_1:InputLayer is as follows: first dimension: 58 frequency bins of theFourier transform (58 dimensions); second dimension: PPG andaccelerometer (two dimensions); and third dimension: magnitude and phaseof the Fourier transform (two dimensions). Such filters are generallyreferred to as kernels. For example, the kernels in the convolutionallayer are the convolutional filters. The kernel size generally refers tothe width by height of the filter mask. Thus, in the illustratedembodiment, the kernel size of the convolutional layers is (3, 2) and(3, 1). In addition, all activations are ReLU (recurrent linear unit)except for the “dense 1: Dense” layer, which has a softmax activation.Further, in this embodiment, the “dense_1: Dense” layer has an outputdimension of 200 because this layer is producing probability values for200 possible heart rate values ranging, e.g., from 30 to 229. Thus, inthe illustrated embodiment, the model has 5,308 parameters. It shouldfurther be understood by those having ordinary skill in the art that theneural network described herein may have any number of nodes,connections, and/or parameters and that the examples provided by way ofFIGS. 11 and 12 are provided for illustrative purposes only.

Accordingly, and referring back to FIG. 8, as shown at 806, the method800 includes determining the heart rate estimation in beats-per-minuteusing the output (e.g., output 1108) of the computing system 1000corresponding to the heart rate probability distribution for themultiple PPG signals. As shown at 808, the method 800 includes causingthe heart rate estimation to be displayed via a user interface on theuser device. For example, in an embodiment, as mentioned, the processor110 described herein may drive the display 114 to display the generatedHR on a user device (such as a mobile phone or a wearable computingdevice).

Moreover, in an embodiment, each of the heart rate probabilitydistributions for the multiple PPG signals may be combined to form acombined heart rate probability distribution. For example, as shown inFIG. 13A, a graph 1300 including a combination of heart rate estimates1302, 1304, 1306, 1308 and their corresponding probability from multiplechannels at one time point is illustrated. Thus, as shown in FIG. 13B,the combination of the heart rate estimates 1302, 1304, 1306, 1308 andtheir corresponding probability from FIG. 13A can be used to generate asingle estimated heart rate 1310 with an overall higher probability.Thus, in such embodiments, the computing system 1000 may determine apeak 1312 in the combined probability distribution (i.e., the singleestimated heart rate 1310) and use the peak 1312 to select the heartrate estimation.

In still further embodiments, the method 800 may include determining avariance in the heart rate probability distributions for the multiplePPG signals and weighting the heart rate probability distributions basedon their inverse variance. In another embodiment, the heart rateprobability distribution for each of the multiple PPG signals may becombined into a probability distribution combination over successivetime points to allow temporal updates of heart rate that include theconfidence metric. Accordingly, in such embodiments, the probabilitydistribution combination can be weighted such that more recentmeasurements are given more weight than older measurements. In yetanother embodiment, the estimate and variance of the probabilitydistribution can be directly input into a Kalman filter.

For example, as shown in FIG. 14, a graph 1400 of heart rate (y-axis)versus time (x-axis) is illustrated, particularly illustrating the heartrate probability distributions 1402 as output by the machine learningmodel 1006, as well as the combined heart rate estimate 1404. Inparticular, as shown, a curve (representing the combined heart rateestimate 1404) is fit to the heart rate probability distributions ateach time interval (e.g., t₁, t₂, t₃, t₄, t₅, t₆, t₇, t₈, and so on)such that distributions 1402 having a higher confidence level (e.g., t₃,t₅, and t₈ due to the lower variance) are weighted higher than thosewith a lower confidence level (e.g., t₂, t₆, and t₇ due to having ahigher variance). Thus, in this embodiment, the confidence level may beequal to the inverse of the variance, e.g., the probabilities are veryspread out (high variance) over a large range of heart rate values,therefore, the confidence level is low.

Example Embodiments (EEs)

EE 1. 1. A method for generating a physiological metric, the methodcomprising: obtaining a first PPG signal based on light received by alight detector from a first light source during a first temporal windowduring which a second light source is not emitting light; obtaining asecond PPG signal based on light received by the light detector from thesecond light source different from the first light source and during asecond temporal window that is different from the first temporal windowand during which the first light source is not emitting light; removinga motion component from at least one of the first and second PPG signalsbased on information associated with a first distance between the firstlight source and the light detector and a second distance between thesecond light source and the light detector; generating the physiologicalmetric based on at least one of the first PPG signal and the second PPGsignal having the motion component removed therefrom; and causing thephysiological metric to be displayed via a user interface on a userdevice.

EE 2. The method of EE 1, further comprising: processing, via a trainedmachine learning model of the user device, the first and second PPGsignals to produce an output corresponding to a heart rate probabilitydistribution for the first and second PPG signals; and determining thephysiological metric using the output corresponding to the heart rateprobability distribution for the first and second PPG signals.

EE 3. The method of EE 2, wherein the heart rate probabilitydistribution for the first and second PPG signals are combined to form acombined heart rate probability distribution.

EE 4. The method of EE 3, further comprising: determining a peak in thecombined probability distribution; and using the peak to select theheart rate estimation.

EE 5. The method of EE 2, further comprising determining at least oneconfidence metric associated with at least one of first and second PPGsignals, the confidence metric being indicative of a signal quality fora particular light path associated with each of the first and second PPGsignals.

EE 6. The method of EE 5, further comprising: determining a sum ofprobabilities for all heart rates within a fixed number ofbeats-per-minute of the heart rate estimation; and

using the sum as the confidence metric.

EE 7. The method of EE 5, further comprising: determining a variance inthe heart rate probability distributions for the first and second PPGsignals; and weighing the heart rate probability distributions based onthe variance.

EE 8. The method of EE 5, wherein the heart rate probabilitydistribution for each of the first and second PPG signals are combinedinto a probability distribution combination over successive time pointsto allow temporal updates of heart rate that include the confidencemetric.

EE 9. The method of claim 8, wherein the probability distributioncombination is weighted such that more recent measurements are givenmore weight than older measurements.

EE 10. A method for generating a heart rate estimation for a userdevice, the method comprising: obtaining multiple photoplethysmography(PPG) signals based on light received by at least one light detector ofthe user device from multiple light sources during different temporalwindows during which other light sources of the multiple light sourcesare not emitting light, wherein the multiple PPG signals each representa different channel of light received by the at least one light detectorfrom the multiple light sources; processing the multiple PPG signals toproduce an output corresponding to a heart rate probability distributionfor the multiple PPG signals; determining the heart rate estimation inbeats-per-minute using the output corresponding to the heart rateprobability distribution for the multiple PPG signals; and causing theheart rate estimation to be displayed via a user interface on the userdevice.

EE 11. The method of EE 10, further comprising processing the multiplePPG signals to produce the output corresponding to the heart rateprobability distribution for the multiple PPG signals via a trainedmachine learning model of the user device.

EE 12. The method of EE 11, wherein the trained machine learning modelcomprises a neural network.

EE 13. The method of EE 10, wherein the heart rate probabilitydistribution for the multiple PPG signals are combined to form acombined heart rate probability distribution.

EE 14. The method of EE 13, further comprising: determining a peak inthe combined probability distribution; and using the peak to select theheart rate estimation.

EE 15. The method of EE 10, further comprising determining at least oneconfidence metric associated with at least one of multiple PPG signals,the confidence metric being indicative of a signal quality for aparticular light path associated with each of the multiple PPG signals.

EE 16. The method of EE 15, further comprising: determining a sum ofprobabilities for all heart rates within a fixed number ofbeats-per-minute of the heart rate estimation; and using the sum as theconfidence metric.

EE 17. The method of EE 15, further comprising: determining a variancein the heart rate probability distributions for the multiple PPGsignals; and weighing the heart rate probability distributions based onthe variance.

EE 18. The method of claim 14, wherein the heart rate probabilitydistribution for each of the multiple PPG signals are combined into aprobability distribution combination over successive time points toallow temporal updates of heart rate that include the confidence metric.

EE 19. The method of EE 18, wherein the probability distributioncombination is weighted such that more recent measurements are givenmore weight than older measurements.

EE 20. A wearable device for tracking a heart rate estimation of a user,comprising: a plurality of light sources; a light detector; a memory;and one or more processors, wherein the memory storescomputer-executable instructions for causing the one or more processorsto: obtain multiple photoplethysmography (PPG) signals based on lightreceived by the light detector from the multiple light sources duringdifferent temporal windows during which other light sources of themultiple light sources are not emitting light; process, via a trainedmachine learning model of the one or more processors, the multiple PPGsignals to produce an output corresponding to a heart rate probabilitydistribution for the multiple PPG signals; determine the heart rateestimation in beats-per-minute using the multiple PPG signals and theoutput corresponding to the heart rate probability distribution for themultiple PPG signals; and cause the heart rate estimation to bedisplayed via a user interface on the wearable device.

Other Considerations

Information and signals disclosed herein may be represented using any ofa variety of different technologies and techniques. For example, data,instructions, commands, information, signals, bits, symbols, and chipsthat may be referenced throughout the above description may berepresented by voltages, currents, electromagnetic waves, magneticfields or particles, optical fields or particles, or any combinationthereof.

The various illustrative logical blocks, and algorithm steps describedin connection with the embodiments disclosed herein may be implementedas electronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, blocks, and steps have been describedabove generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure.

The techniques described herein may be implemented in hardware,software, firmware, or any combination thereof. Such techniques may beimplemented in any of a variety of devices, such as, for example,wearable devices, wireless communication device handsets, or integratedcircuit devices for wearable devices, wireless communication devicehandsets, and other devices. Any features described as devices orcomponents may be implemented together in an integrated logic device orseparately as discrete but interoperable logic devices. If implementedin software, the techniques may be realized at least in part by acomputer-readable data storage medium comprising program code includinginstructions that, when executed, performs one or more of the methodsdescribed above. The computer-readable data storage medium may form partof a computer program product, which may include packaging materials.The computer-readable medium may comprise memory or data storage media,such as random access memory (RAM) such as synchronous dynamic randomaccess memory (SDRAM), read-only memory (ROM), non-volatile randomaccess memory (NVRAM), electrically erasable programmable read-onlymemory (EEPROM), FLASH memory, magnetic or optical data storage media,and the like. The techniques additionally, or alternatively, may berealized at least in part by a computer-readable communication mediumthat carries or communicates program code in the form of instructions ordata structures and that can be accessed, read, and/or executed by acomputer, such as propagated signals or waves.

According to some embodiments, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices, wearable devices, or any other device thatincorporates hard-wired and/or program logic to implement thetechniques.

Processor(s) in communication with (e.g., operating in collaborationwith) the computer-readable medium (e.g., memory or other data storagedevice) may execute instructions of the program code, and may includeone or more processors, such as one or more digital signal processors(DSPs), general purpose microprocessors, ASICs, FPGAs, or otherequivalent integrated or discrete logic circuitry. Such a processor maybe configured to perform any of the techniques described in thisdisclosure. A general purpose processor may be a microprocessor; but inthe alternative, the processor may be any conventional processor,controller, microcontroller, or state machine. A processor may also beimplemented as a combination of computing devices, for example, acombination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Accordingly, the term“processor,” as used herein may refer to any of the foregoing structure,any combination of the foregoing structure, or any other structure orapparatus suitable for implementation of the techniques describedherein. Also, the techniques could be fully implemented in one or morecircuits or logic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wearable device, a wirelesshandset, an integrated circuit (IC) or a set of ICs (e.g., a chip set).Various components, or units are described in this disclosure toemphasize functional aspects of devices configured to perform thedisclosed techniques, but do not necessarily require realization bydifferent hardware units. Rather, as described above, various units maybe combined in a hardware unit or provided by a collection ofinter-operative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

Although the foregoing has been described in connection with variousdifferent embodiments, features or elements from one embodiment may becombined with other embodiments without departing from the teachings ofthis disclosure.

What is claimed is:
 1. A method for generating a physiological metric,the method comprising: obtaining a first PPG signal based on lightreceived by a light detector from a first light source during a firsttemporal window during which a second light source is not emittinglight; obtaining a second PPG signal based on light received by thelight detector from the second light source different from the firstlight source and during a second temporal window that is different fromthe first temporal window and during which the first light source is notemitting light; removing a motion component from at least one of thefirst and second PPG signals based on information associated with afirst distance between the first light source and the light detector anda second distance between the second light source and the lightdetector; generating the physiological metric based on at least one ofthe first PPG signal and the second PPG signal having the motioncomponent removed therefrom; and causing the physiological metric to bedisplayed via a user interface on a user device.
 2. The method of claim1, further comprising: processing, via a trained machine learning modelof the user device, the first and second PPG signals to produce anoutput corresponding to a heart rate probability distribution for thefirst and second PPG signals; and determining the physiological metricusing the output corresponding to the heart rate probabilitydistribution for the first and second PPG signals.
 3. The method ofclaim 2, wherein the heart rate probability distribution for the firstand second PPG signals are combined to form a combined heart rateprobability distribution.
 4. The method of claim 3, further comprising:determining a peak in the combined probability distribution; and usingthe peak to select the heart rate estimation.
 5. The method of claim 2,further comprising determining at least one confidence metric associatedwith at least one of first and second PPG signals, the confidence metricbeing indicative of a signal quality for a particular light pathassociated with each of the first and second PPG signals.
 6. The methodof claim 5, further comprising: determining a sum of probabilities forall heart rates within a fixed number of beats-per-minute of the heartrate estimation; and using the sum as the confidence metric.
 7. Themethod of claim 5, further comprising: determining a variance in theheart rate probability distributions for the first and second PPGsignals; and weighing the heart rate probability distributions based onthe variance.
 8. The method of claim 5, wherein the heart rateprobability distribution for each of the first and second PPG signalsare combined into a probability distribution combination over successivetime points to allow temporal updates of heart rate that include theconfidence metric.
 9. The method of claim 8, wherein the probabilitydistribution combination is weighted such that more recent measurementsare given more weight than older measurements.
 10. A method forgenerating a heart rate estimation for a user device, the methodcomprising: obtaining multiple photoplethysmography (PPG) signals basedon light received by at least one light detector of the user device frommultiple light sources during different temporal windows during whichother light sources of the multiple light sources are not emittinglight, wherein the multiple PPG signals each represent a differentchannel of light received by the at least one light detector from themultiple light sources; processing the multiple PPG signals to producean output corresponding to a heart rate probability distribution for themultiple PPG signals; determining the heart rate estimation inbeats-per-minute using the output corresponding to the heart rateprobability distribution for the multiple PPG signals; and causing theheart rate estimation to be displayed via a user interface on the userdevice.
 11. The method of claim 10, further comprising processing themultiple PPG signals to produce the output corresponding to the heartrate probability distribution for the multiple PPG signals via a trainedmachine learning model of the user device.
 12. The method of claim 11,wherein the trained machine learning model comprises a neural network.13. The method of claim 10, wherein the heart rate probabilitydistribution for the multiple PPG signals are combined to form acombined heart rate probability distribution.
 14. The method of claim13, further comprising: determining a peak in the combined probabilitydistribution; and using the peak to select the heart rate estimation.15. The method of claim 10, further comprising determining at least oneconfidence metric associated with at least one of multiple PPG signals,the confidence metric being indicative of a signal quality for aparticular light path associated with each of the multiple PPG signals.16. The method of claim 15, further comprising: determining a sum ofprobabilities for all heart rates within a fixed number ofbeats-per-minute of the heart rate estimation; and using the sum as theconfidence metric.
 17. The method of claim 15, further comprising:determining a variance in the heart rate probability distributions forthe multiple PPG signals; and weighing the heart rate probabilitydistributions based on the variance.
 18. The method of claim 14, whereinthe heart rate probability distribution for each of the multiple PPGsignals are combined into a probability distribution combination oversuccessive time points to allow temporal updates of heart rate thatinclude the confidence metric.
 19. The method of claim 18, wherein theprobability distribution combination is weighted such that more recentmeasurements are given more weight than older measurements.
 20. Awearable device for tracking a heart rate estimation of a user,comprising: a plurality of light sources; a light detector; a memory;and one or more processors, wherein the memory storescomputer-executable instructions for causing the one or more processorsto: obtain multiple photoplethysmography (PPG) signals based on lightreceived by the light detector from the multiple light sources duringdifferent temporal windows during which other light sources of themultiple light sources are not emitting light; process, via a trainedmachine learning model of the one or more processors, the multiple PPGsignals to produce an output corresponding to a heart rate probabilitydistribution for the multiple PPG signals; determine the heart rateestimation in beats-per-minute using the multiple PPG signals and theoutput corresponding to the heart rate probability distribution for themultiple PPG signals; and cause the heart rate estimation to bedisplayed via a user interface on the wearable device.